Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Use brackets to get non-tensor data in gym envs #2769

Open
wants to merge 1 commit into
base: gh/vmoens/86/base
Choose a base branch
from
Open

[BugFix] Use brackets to get non-tensor data in gym envs #2769

wants to merge 1 commit into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 7, 2025 •
edited
Loading

@vmoens
Update
87a342c 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 7, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2769

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 10 Unrelated Failures

As of commit 87a342c with merge base 75f113f (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens mentioned this pull request Feb 7, 2025
Open
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 7, 2025
@vmoens vmoens mentioned this pull request Feb 7, 2025
Open
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.5416s 0.4540s 2.2026 Ops/s 2.2069 Ops/s $\color{#d91a1a}-0.19\%$
test_transformed 1.0191s 0.9318s 1.0732 Ops/s 1.0791 Ops/s $\color{#d91a1a}-0.54\%$
test_serial 1.4684s 1.3888s 0.7200 Ops/s 0.7227 Ops/s $\color{#d91a1a}-0.36\%$
test_parallel 1.2911s 1.2090s 0.8271 Ops/s 0.7980 Ops/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[True-True-True-True-True] 0.3854ms 30.2571μs 33.0501 KOps/s 33.2850 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-True-True-True-False] 0.1092ms 17.9934μs 55.5758 KOps/s 56.3538 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[True-True-True-False-True] 64.6610μs 17.0605μs 58.6149 KOps/s 59.5406 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[True-True-True-False-False] 33.2930μs 10.1644μs 98.3824 KOps/s 100.0814 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-True-False-True-True] 86.9430μs 32.3985μs 30.8656 KOps/s 31.0709 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[True-True-False-True-False] 44.1230μs 19.9976μs 50.0059 KOps/s 51.2432 KOps/s $\color{#d91a1a}-2.41\%$
test_step_mdp_speed[True-True-False-False-True] 49.3330μs 18.8492μs 53.0526 KOps/s 53.1804 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-False-False-False] 47.3990μs 12.0988μs 82.6528 KOps/s 84.5376 KOps/s $\color{#d91a1a}-2.23\%$
test_step_mdp_speed[True-False-True-True-True] 77.2450μs 34.3436μs 29.1175 KOps/s 29.8550 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-False-True-True-False] 63.7390μs 21.9024μs 45.6571 KOps/s 46.4753 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-False-True-False-True] 0.6043ms 18.9199μs 52.8543 KOps/s 53.4754 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-False-True-False-False] 69.6040μs 12.0268μs 83.1475 KOps/s 85.6086 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[True-False-False-True-True] 97.6830μs 36.1987μs 27.6253 KOps/s 28.5197 KOps/s $\color{#d91a1a}-3.14\%$
test_step_mdp_speed[True-False-False-True-False] 57.6570μs 23.8323μs 41.9599 KOps/s 43.1641 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[True-False-False-False-True] 55.7440μs 20.7916μs 48.0963 KOps/s 49.1126 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-False-False-False-False] 33.1630μs 13.8431μs 72.2382 KOps/s 73.4567 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-True-True-True-True] 83.0560μs 34.3398μs 29.1207 KOps/s 29.4795 KOps/s $\color{#d91a1a}-1.22\%$
test_step_mdp_speed[False-True-True-True-False] 73.0170μs 21.9982μs 45.4582 KOps/s 46.3851 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[False-True-True-False-True] 65.1530μs 21.6841μs 46.1168 KOps/s 47.0479 KOps/s $\color{#d91a1a}-1.98\%$
test_step_mdp_speed[False-True-True-False-False] 65.2820μs 13.4365μs 74.4243 KOps/s 74.8181 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-True-False-True-True] 75.1210μs 35.3336μs 28.3016 KOps/s 28.4163 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-True-False] 67.5970μs 23.7152μs 42.1671 KOps/s 43.3758 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-True-False-False-True] 2.7910ms 24.0141μs 41.6423 KOps/s 43.3267 KOps/s $\color{#d91a1a}-3.89\%$
test_step_mdp_speed[False-True-False-False-False] 35.4560μs 15.3129μs 65.3043 KOps/s 67.1087 KOps/s $\color{#d91a1a}-2.69\%$
test_step_mdp_speed[False-False-True-True-True] 93.3850μs 37.7623μs 26.4814 KOps/s 27.1904 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-True-True-False] 53.6200μs 25.5886μs 39.0799 KOps/s 40.0158 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-False-True-False-True] 71.4250μs 23.4126μs 42.7121 KOps/s 43.5596 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-False-True-False-False] 63.2480μs 15.3908μs 64.9740 KOps/s 67.0025 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-False-False-True-True] 0.6124ms 38.7861μs 25.7824 KOps/s 26.0886 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-False-False-True-False] 74.3890μs 27.5094μs 36.3513 KOps/s 37.5194 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[False-False-False-False-True] 0.1572ms 24.9267μs 40.1177 KOps/s 40.6241 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-False-False-False-False] 0.1754ms 16.9456μs 59.0123 KOps/s 61.0407 KOps/s $\color{#d91a1a}-3.32\%$
test_values[generalized_advantage_estimate-True-True] 12.3704ms 9.3530ms 106.9181 Ops/s 100.7214 Ops/s $\textbf{\color{#35bf28}+6.15\%}$
test_values[vec_generalized_advantage_estimate-True-True] 31.0151ms 26.8770ms 37.2066 Ops/s 36.8571 Ops/s $\color{#35bf28}+0.95\%$
test_values[td0_return_estimate-False-False] 0.2431ms 0.1810ms 5.5259 KOps/s 5.1456 KOps/s $\textbf{\color{#35bf28}+7.39\%}$
test_values[td1_return_estimate-False-False] 23.4694ms 23.0139ms 43.4520 Ops/s 41.2176 Ops/s $\textbf{\color{#35bf28}+5.42\%}$
test_values[vec_td1_return_estimate-False-False] 28.7087ms 26.9297ms 37.1337 Ops/s 36.9905 Ops/s $\color{#35bf28}+0.39\%$
test_values[td_lambda_return_estimate-True-False] 36.3140ms 33.0279ms 30.2775 Ops/s 28.7108 Ops/s $\textbf{\color{#35bf28}+5.46\%}$
test_values[vec_td_lambda_return_estimate-True-False] 29.0054ms 26.9497ms 37.1062 Ops/s 36.6452 Ops/s $\color{#35bf28}+1.26\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2809ms 8.1779ms 122.2811 Ops/s 119.8056 Ops/s $\color{#35bf28}+2.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4105ms 1.8711ms 534.4345 Ops/s 512.3851 Ops/s $\color{#35bf28}+4.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5610ms 0.3589ms 2.7865 KOps/s 2.7651 KOps/s $\color{#35bf28}+0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.8893ms 46.8536ms 21.3431 Ops/s 21.6330 Ops/s $\color{#d91a1a}-1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.3826ms 3.4221ms 292.2184 Ops/s 288.6485 Ops/s $\color{#35bf28}+1.24\%$
test_dqn_speed[False-None] 6.0792ms 1.4089ms 709.7674 Ops/s 692.0482 Ops/s $\color{#35bf28}+2.56\%$
test_dqn_speed[False-backward] 2.0268ms 1.8918ms 528.6078 Ops/s 515.7995 Ops/s $\color{#35bf28}+2.48\%$
test_dqn_speed[True-None] 0.6681ms 0.4755ms 2.1031 KOps/s 2.0702 KOps/s $\color{#35bf28}+1.59\%$
test_dqn_speed[True-backward] 0.9643ms 0.8987ms 1.1127 KOps/s 1.0635 KOps/s $\color{#35bf28}+4.62\%$
test_dqn_speed[reduce-overhead-None] 0.5869ms 0.4749ms 2.1055 KOps/s 2.0449 KOps/s $\color{#35bf28}+2.97\%$
test_dqn_speed[reduce-overhead-backward] 0.9395ms 0.8943ms 1.1182 KOps/s 1.0868 KOps/s $\color{#35bf28}+2.88\%$
test_ddpg_speed[False-None] 3.6938ms 2.8837ms 346.7745 Ops/s 342.1530 Ops/s $\color{#35bf28}+1.35\%$
test_ddpg_speed[False-backward] 4.9319ms 4.1818ms 239.1302 Ops/s 242.2135 Ops/s $\color{#d91a1a}-1.27\%$
test_ddpg_speed[True-None] 1.9206ms 1.2232ms 817.5072 Ops/s 798.1042 Ops/s $\color{#35bf28}+2.43\%$
test_ddpg_speed[True-backward] 2.2733ms 2.1043ms 475.2136 Ops/s 411.4769 Ops/s $\textbf{\color{#35bf28}+15.49\%}$
test_ddpg_speed[reduce-overhead-None] 1.7014ms 1.2235ms 817.3342 Ops/s 814.8504 Ops/s $\color{#35bf28}+0.30\%$
test_ddpg_speed[reduce-overhead-backward] 2.2736ms 2.1746ms 459.8647 Ops/s 462.2132 Ops/s $\color{#d91a1a}-0.51\%$
test_sac_speed[False-None] 9.9081ms 8.0897ms 123.6135 Ops/s 122.8598 Ops/s $\color{#35bf28}+0.61\%$
test_sac_speed[False-backward] 12.7017ms 10.9490ms 91.3322 Ops/s 91.6879 Ops/s $\color{#d91a1a}-0.39\%$
test_sac_speed[True-None] 2.4130ms 2.1382ms 467.6862 Ops/s 474.9288 Ops/s $\color{#d91a1a}-1.52\%$
test_sac_speed[True-backward] 3.8168ms 3.7586ms 266.0568 Ops/s 266.1458 Ops/s $\color{#d91a1a}-0.03\%$
test_sac_speed[reduce-overhead-None] 2.3615ms 2.0914ms 478.1418 Ops/s 472.2154 Ops/s $\color{#35bf28}+1.26\%$
test_sac_speed[reduce-overhead-backward] 4.0613ms 3.9240ms 254.8399 Ops/s 258.1300 Ops/s $\color{#d91a1a}-1.27\%$
test_redq_speed[False-None] 17.9426ms 13.6867ms 73.0635 Ops/s 72.6201 Ops/s $\color{#35bf28}+0.61\%$
test_redq_speed[False-backward] 28.3629ms 23.3094ms 42.9011 Ops/s 44.0272 Ops/s $\color{#d91a1a}-2.56\%$
test_redq_speed[True-None] 6.1137ms 5.4658ms 182.9564 Ops/s 190.3367 Ops/s $\color{#d91a1a}-3.88\%$
test_redq_speed[True-backward] 15.1598ms 13.4921ms 74.1175 Ops/s 77.3076 Ops/s $\color{#d91a1a}-4.13\%$
test_redq_speed[reduce-overhead-None] 6.5588ms 5.7191ms 174.8531 Ops/s 190.4910 Ops/s $\textbf{\color{#d91a1a}-8.21\%}$
test_redq_speed[reduce-overhead-backward] 15.8427ms 13.2033ms 75.7384 Ops/s 77.3780 Ops/s $\color{#d91a1a}-2.12\%$
test_redq_deprec_speed[False-None] 14.4946ms 13.2295ms 75.5887 Ops/s 75.2883 Ops/s $\color{#35bf28}+0.40\%$
test_redq_deprec_speed[False-backward] 20.9109ms 18.5613ms 53.8755 Ops/s 52.4640 Ops/s $\color{#35bf28}+2.69\%$
test_redq_deprec_speed[True-None] 5.4488ms 4.2352ms 236.1173 Ops/s 254.2389 Ops/s $\textbf{\color{#d91a1a}-7.13\%}$
test_redq_deprec_speed[True-backward] 9.1560ms 8.9388ms 111.8716 Ops/s 118.9836 Ops/s $\textbf{\color{#d91a1a}-5.98\%}$
test_redq_deprec_speed[reduce-overhead-None] 5.0690ms 4.1342ms 241.8831 Ops/s 256.0000 Ops/s $\textbf{\color{#d91a1a}-5.51\%}$
test_redq_deprec_speed[reduce-overhead-backward] 9.7622ms 9.3528ms 106.9201 Ops/s 105.3670 Ops/s $\color{#35bf28}+1.47\%$
test_td3_speed[False-None] 8.7350ms 8.3161ms 120.2481 Ops/s 121.4484 Ops/s $\color{#d91a1a}-0.99\%$
test_td3_speed[False-backward] 11.2897ms 10.6340ms 94.0379 Ops/s 93.7232 Ops/s $\color{#35bf28}+0.34\%$
test_td3_speed[True-None] 2.2764ms 1.9144ms 522.3608 Ops/s 537.8400 Ops/s $\color{#d91a1a}-2.88\%$
test_td3_speed[True-backward] 3.4782ms 3.4192ms 292.4629 Ops/s 288.6155 Ops/s $\color{#35bf28}+1.33\%$
test_td3_speed[reduce-overhead-None] 1.9131ms 1.8322ms 545.7828 Ops/s 527.9760 Ops/s $\color{#35bf28}+3.37\%$
test_td3_speed[reduce-overhead-backward] 3.9948ms 3.6137ms 276.7280 Ops/s 291.2082 Ops/s $\color{#d91a1a}-4.97\%$
test_cql_speed[False-None] 38.5175ms 37.2125ms 26.8727 Ops/s 27.0844 Ops/s $\color{#d91a1a}-0.78\%$
test_cql_speed[False-backward] 53.2528ms 48.0478ms 20.8126 Ops/s 20.9412 Ops/s $\color{#d91a1a}-0.61\%$
test_cql_speed[True-None] 18.3432ms 16.9959ms 58.8378 Ops/s 62.0039 Ops/s $\textbf{\color{#d91a1a}-5.11\%}$
test_cql_speed[True-backward] 28.3656ms 23.7767ms 42.0580 Ops/s 44.0478 Ops/s $\color{#d91a1a}-4.52\%$
test_cql_speed[reduce-overhead-None] 19.0737ms 17.1705ms 58.2395 Ops/s 61.7930 Ops/s $\textbf{\color{#d91a1a}-5.75\%}$
test_cql_speed[reduce-overhead-backward] 25.9980ms 23.7479ms 42.1090 Ops/s 43.2580 Ops/s $\color{#d91a1a}-2.66\%$
test_a2c_speed[False-None] 8.8543ms 7.5552ms 132.3584 Ops/s 138.8280 Ops/s $\color{#d91a1a}-4.66\%$
test_a2c_speed[False-backward] 15.3677ms 14.8546ms 67.3194 Ops/s 69.0209 Ops/s $\color{#d91a1a}-2.47\%$
test_a2c_speed[True-None] 4.4876ms 3.7574ms 266.1440 Ops/s 267.5185 Ops/s $\color{#d91a1a}-0.51\%$
test_a2c_speed[True-backward] 11.6394ms 10.5133ms 95.1179 Ops/s 97.0023 Ops/s $\color{#d91a1a}-1.94\%$
test_a2c_speed[reduce-overhead-None] 4.1226ms 3.7938ms 263.5892 Ops/s 266.7102 Ops/s $\color{#d91a1a}-1.17\%$
test_a2c_speed[reduce-overhead-backward] 11.5170ms 10.4384ms 95.8000 Ops/s 98.0832 Ops/s $\color{#d91a1a}-2.33\%$
test_ppo_speed[False-None] 9.0654ms 7.6126ms 131.3612 Ops/s 133.0303 Ops/s $\color{#d91a1a}-1.25\%$
test_ppo_speed[False-backward] 16.9541ms 15.2221ms 65.6938 Ops/s 67.7048 Ops/s $\color{#d91a1a}-2.97\%$
test_ppo_speed[True-None] 5.9587ms 4.5671ms 218.9596 Ops/s 240.9339 Ops/s $\textbf{\color{#d91a1a}-9.12\%}$
test_ppo_speed[True-backward] 10.3270ms 9.9924ms 100.0757 Ops/s 95.6630 Ops/s $\color{#35bf28}+4.61\%$
test_ppo_speed[reduce-overhead-None] 4.9401ms 4.0748ms 245.4089 Ops/s 240.3212 Ops/s $\color{#35bf28}+2.12\%$
test_ppo_speed[reduce-overhead-backward] 12.0246ms 10.2173ms 97.8732 Ops/s 99.4661 Ops/s $\color{#d91a1a}-1.60\%$
test_reinforce_speed[False-None] 8.5675ms 6.5837ms 151.8909 Ops/s 151.0934 Ops/s $\color{#35bf28}+0.53\%$
test_reinforce_speed[False-backward] 10.4894ms 9.7515ms 102.5479 Ops/s 101.4025 Ops/s $\color{#35bf28}+1.13\%$
test_reinforce_speed[True-None] 3.8418ms 3.1260ms 319.8931 Ops/s 320.0388 Ops/s $\color{#d91a1a}-0.05\%$
test_reinforce_speed[True-backward] 9.4899ms 9.0265ms 110.7853 Ops/s 106.3460 Ops/s $\color{#35bf28}+4.17\%$
test_reinforce_speed[reduce-overhead-None] 3.7696ms 3.0618ms 326.6050 Ops/s 323.7186 Ops/s $\color{#35bf28}+0.89\%$
test_reinforce_speed[reduce-overhead-backward] 9.5101ms 8.9517ms 111.7104 Ops/s 109.8174 Ops/s $\color{#35bf28}+1.72\%$
test_iql_speed[False-None] 33.7186ms 32.0778ms 31.1742 Ops/s 30.2256 Ops/s $\color{#35bf28}+3.14\%$
test_iql_speed[False-backward] 46.7033ms 44.8783ms 22.2825 Ops/s 21.8753 Ops/s $\color{#35bf28}+1.86\%$
test_iql_speed[True-None] 14.2522ms 11.1735ms 89.4978 Ops/s 88.9316 Ops/s $\color{#35bf28}+0.64\%$
test_iql_speed[True-backward] 26.9613ms 22.0839ms 45.2819 Ops/s 44.2290 Ops/s $\color{#35bf28}+2.38\%$
test_iql_speed[reduce-overhead-None] 12.8188ms 11.1602ms 89.6042 Ops/s 87.9994 Ops/s $\color{#35bf28}+1.82\%$
test_iql_speed[reduce-overhead-backward] 23.3268ms 21.7935ms 45.8852 Ops/s 44.6298 Ops/s $\color{#35bf28}+2.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1426ms 4.7996ms 208.3521 Ops/s 207.3980 Ops/s $\color{#35bf28}+0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7239ms 0.5074ms 1.9708 KOps/s 1.9582 KOps/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8132ms 0.4906ms 2.0383 KOps/s 2.0468 KOps/s $\color{#d91a1a}-0.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1044ms 4.5364ms 220.4403 Ops/s 213.1952 Ops/s $\color{#35bf28}+3.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.6313ms 0.5008ms 1.9966 KOps/s 1.9749 KOps/s $\color{#35bf28}+1.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7667ms 0.4794ms 2.0858 KOps/s 2.0929 KOps/s $\color{#d91a1a}-0.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.3030ms 1.6293ms 613.7610 Ops/s 599.0000 Ops/s $\color{#35bf28}+2.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2928ms 1.5464ms 646.6694 Ops/s 629.8005 Ops/s $\color{#35bf28}+2.68\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1115ms 4.7114ms 212.2525 Ops/s 206.8487 Ops/s $\color{#35bf28}+2.61\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.9854ms 0.6475ms 1.5443 KOps/s 1.4871 KOps/s $\color{#35bf28}+3.85\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9286ms 0.6245ms 1.6013 KOps/s 1.6101 KOps/s $\color{#d91a1a}-0.54\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7093ms 4.5369ms 220.4167 Ops/s 213.1388 Ops/s $\color{#35bf28}+3.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.3749ms 0.5157ms 1.9392 KOps/s 1.9176 KOps/s $\color{#35bf28}+1.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7217ms 0.4874ms 2.0516 KOps/s 2.0444 KOps/s $\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.3498ms 4.5380ms 220.3608 Ops/s 214.1442 Ops/s $\color{#35bf28}+2.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0878ms 0.5100ms 1.9608 KOps/s 1.9836 KOps/s $\color{#d91a1a}-1.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6935ms 0.4751ms 2.1049 KOps/s 2.0287 KOps/s $\color{#35bf28}+3.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.6651ms 4.6971ms 212.8953 Ops/s 209.4330 Ops/s $\color{#35bf28}+1.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.4646s 1.2998ms 769.3583 Ops/s 1.5421 KOps/s $\textbf{\color{#d91a1a}-50.11\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8092ms 0.6172ms 1.6202 KOps/s 1.5973 KOps/s $\color{#35bf28}+1.43\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 8.7937ms 4.4496ms 224.7374 Ops/s 239.3892 Ops/s $\textbf{\color{#d91a1a}-6.12\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.3337ms 2.3462ms 426.2194 Ops/s 435.9993 Ops/s $\color{#d91a1a}-2.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.7271ms 1.2872ms 776.8896 Ops/s 777.4949 Ops/s $\color{#d91a1a}-0.08\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.6294ms 4.2144ms 237.2820 Ops/s 242.4394 Ops/s $\color{#d91a1a}-2.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 5.0405ms 2.2992ms 434.9351 Ops/s 438.2259 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.6125ms 1.3944ms 717.1639 Ops/s 704.1008 Ops/s $\color{#35bf28}+1.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4112s 12.6168ms 79.2593 Ops/s 237.1669 Ops/s $\textbf{\color{#d91a1a}-66.58\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.0819ms 2.4876ms 401.9912 Ops/s 411.5519 Ops/s $\color{#d91a1a}-2.32\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.1003ms 1.3813ms 723.9475 Ops/s 651.3575 Ops/s $\textbf{\color{#35bf28}+11.14\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 12.1296ms 11.8881ms 84.1178 Ops/s 80.2028 Ops/s $\color{#35bf28}+4.88\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 16.8438ms 14.2487ms 70.1819 Ops/s 71.5460 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 21.3772ms 20.7502ms 48.1923 Ops/s 46.3922 Ops/s $\color{#35bf28}+3.88\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.0883ms 14.4354ms 69.2743 Ops/s 68.5589 Ops/s $\color{#35bf28}+1.04\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 21.4639ms 20.6674ms 48.3854 Ops/s 46.9010 Ops/s $\color{#35bf28}+3.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.5738ms 15.6688ms 63.8211 Ops/s 63.3762 Ops/s $\color{#35bf28}+0.70\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

Result of GPU Benchmark Tests

Expand to view detailed results
Name Max Mean Ops
test_simple 0.8463s 0.7581s 1.3190 Ops/s
test_transformed 1.3381s 1.3280s 0.7530 Ops/s
test_serial 2.1700s 2.1684s 0.4612 Ops/s
test_parallel 1.8799s 1.8448s 0.5421 Ops/s
test_step_mdp_speed[True-True-True-True-True] 0.4439ms 41.1062μs 24.3272 KOps/s
test_step_mdp_speed[True-True-True-True-False] 62.2610μs 24.3089μs 41.1372 KOps/s
test_step_mdp_speed[True-True-True-False-True] 82.6110μs 22.9778μs 43.5203 KOps/s
test_step_mdp_speed[True-True-True-False-False] 40.4810μs 13.5658μs 73.7150 KOps/s
test_step_mdp_speed[True-True-False-True-True] 86.0320μs 44.0264μs 22.7136 KOps/s
test_step_mdp_speed[True-True-False-True-False] 58.3010μs 26.6075μs 37.5835 KOps/s
test_step_mdp_speed[True-True-False-False-True] 59.2000μs 25.8362μs 38.7053 KOps/s
test_step_mdp_speed[True-True-False-False-False] 44.9100μs 16.0188μs 62.4265 KOps/s
test_step_mdp_speed[True-False-True-True-True] 86.4010μs 45.6784μs 21.8922 KOps/s
test_step_mdp_speed[True-False-True-True-False] 62.8710μs 29.4386μs 33.9690 KOps/s
test_step_mdp_speed[True-False-True-False-True] 98.5110μs 25.7652μs 38.8120 KOps/s
test_step_mdp_speed[True-False-True-False-False] 51.5500μs 15.8713μs 63.0070 KOps/s
test_step_mdp_speed[True-False-False-True-True] 97.0320μs 49.2226μs 20.3159 KOps/s
test_step_mdp_speed[True-False-False-True-False] 60.6010μs 31.5056μs 31.7404 KOps/s
test_step_mdp_speed[True-False-False-False-True] 82.8710μs 27.5280μs 36.3266 KOps/s
test_step_mdp_speed[True-False-False-False-False] 45.6300μs 18.5807μs 53.8194 KOps/s
test_step_mdp_speed[False-True-True-True-True] 84.4610μs 47.2363μs 21.1702 KOps/s
test_step_mdp_speed[False-True-True-True-False] 64.1810μs 29.8230μs 33.5312 KOps/s
test_step_mdp_speed[False-True-True-False-True] 2.5320ms 30.8578μs 32.4067 KOps/s
test_step_mdp_speed[False-True-True-False-False] 52.2510μs 17.8947μs 55.8824 KOps/s
test_step_mdp_speed[False-True-False-True-True] 82.6320μs 49.6533μs 20.1396 KOps/s
test_step_mdp_speed[False-True-False-True-False] 59.7010μs 32.0520μs 31.1993 KOps/s
test_step_mdp_speed[False-True-False-False-True] 0.1044ms 33.0045μs 30.2989 KOps/s
test_step_mdp_speed[False-True-False-False-False] 53.6900μs 20.2523μs 49.3772 KOps/s
test_step_mdp_speed[False-False-True-True-True] 85.2010μs 50.9491μs 19.6274 KOps/s
test_step_mdp_speed[False-False-True-True-False] 90.1720μs 33.4623μs 29.8843 KOps/s
test_step_mdp_speed[False-False-True-False-True] 0.2007ms 31.7342μs 31.5117 KOps/s
test_step_mdp_speed[False-False-True-False-False] 57.4610μs 20.1509μs 49.6256 KOps/s
test_step_mdp_speed[False-False-False-True-True] 0.1007ms 54.1949μs 18.4519 KOps/s
test_step_mdp_speed[False-False-False-True-False] 70.2510μs 37.0081μs 27.0211 KOps/s
test_step_mdp_speed[False-False-False-False-True] 67.7100μs 34.4388μs 29.0370 KOps/s
test_step_mdp_speed[False-False-False-False-False] 60.5510μs 22.8180μs 43.8251 KOps/s
test_values[generalized_advantage_estimate-True-True] 25.7344ms 25.3065ms 39.5155 Ops/s
test_values[vec_generalized_advantage_estimate-True-True] 0.1002s 2.9060ms 344.1168 Ops/s
test_values[td0_return_estimate-False-False] 0.1070ms 80.2513μs 12.4609 KOps/s
test_values[td1_return_estimate-False-False] 56.9991ms 56.2885ms 17.7656 Ops/s
test_values[vec_td1_return_estimate-False-False] 1.3446ms 1.0866ms 920.3263 Ops/s
test_values[td_lambda_return_estimate-True-False] 90.0708ms 88.8577ms 11.2539 Ops/s
test_values[vec_td_lambda_return_estimate-True-False] 1.3566ms 1.0958ms 912.5495 Ops/s
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.4608ms 25.0351ms 39.9438 Ops/s
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0315ms 0.7571ms 1.3208 KOps/s
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8035ms 0.6760ms 1.4793 KOps/s
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6335ms 1.4897ms 671.2937 Ops/s
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7280ms 0.6878ms 1.4540 KOps/s
test_dqn_speed[False-None] 6.7060ms 1.5398ms 649.4395 Ops/s
test_dqn_speed[False-backward] 2.2459ms 2.1525ms 464.5669 Ops/s
test_dqn_speed[True-None] 0.7687ms 0.5616ms 1.7808 KOps/s
test_dqn_speed[True-backward] 1.2984ms 1.2320ms 811.6862 Ops/s
test_dqn_speed[reduce-overhead-None] 0.7393ms 0.5755ms 1.7376 KOps/s
test_dqn_speed[reduce-overhead-backward] 1.1118ms 1.0712ms 933.4954 Ops/s
test_ddpg_speed[False-None] 3.2607ms 2.9292ms 341.3941 Ops/s
test_ddpg_speed[False-backward] 4.6608ms 4.3126ms 231.8807 Ops/s
test_ddpg_speed[True-None] 1.5491ms 1.3586ms 736.0501 Ops/s
test_ddpg_speed[True-backward] 2.6735ms 2.5954ms 385.2903 Ops/s
test_ddpg_speed[reduce-overhead-None] 1.5184ms 1.3682ms 730.8805 Ops/s
test_ddpg_speed[reduce-overhead-backward] 2.1608ms 2.0714ms 482.7547 Ops/s
test_sac_speed[False-None] 8.6358ms 8.2096ms 121.8089 Ops/s
test_sac_speed[False-backward] 11.8651ms 11.2710ms 88.7229 Ops/s
test_sac_speed[True-None] 2.0001ms 1.8570ms 538.5061 Ops/s
test_sac_speed[True-backward] 3.9024ms 3.7342ms 267.7950 Ops/s
test_sac_speed[reduce-overhead-None] 21.9811ms 12.3264ms 81.1268 Ops/s
test_sac_speed[reduce-overhead-backward] 1.8475ms 1.7889ms 558.9941 Ops/s
test_redq_speed[False-None] 8.0990ms 7.6230ms 131.1817 Ops/s
test_redq_speed[False-backward] 12.2377ms 11.7136ms 85.3706 Ops/s
test_redq_speed[True-None] 2.4950ms 2.3096ms 432.9765 Ops/s
test_redq_speed[True-backward] 4.2395ms 4.1627ms 240.2285 Ops/s
test_redq_speed[reduce-overhead-None] 2.4890ms 2.3236ms 430.3604 Ops/s
test_redq_speed[reduce-overhead-backward] 4.3457ms 4.1931ms 238.4865 Ops/s
test_redq_deprec_speed[False-None] 9.5451ms 9.1522ms 109.2630 Ops/s
test_redq_deprec_speed[False-backward] 13.2801ms 12.3674ms 80.8576 Ops/s
test_redq_deprec_speed[True-None] 2.8012ms 2.6537ms 376.8285 Ops/s
test_redq_deprec_speed[True-backward] 4.8694ms 4.4917ms 222.6307 Ops/s
test_redq_deprec_speed[reduce-overhead-None] 2.8121ms 2.6563ms 376.4596 Ops/s
test_redq_deprec_speed[reduce-overhead-backward] 4.7044ms 4.5111ms 221.6756 Ops/s
test_td3_speed[False-None] 8.4091ms 8.0955ms 123.5250 Ops/s
test_td3_speed[False-backward] 11.3823ms 10.6218ms 94.1464 Ops/s
test_td3_speed[True-None] 1.7601ms 1.6697ms 598.9231 Ops/s
test_td3_speed[True-backward] 3.6303ms 3.3692ms 296.8106 Ops/s
test_td3_speed[reduce-overhead-None] 52.6135ms 27.1406ms 36.8452 Ops/s
test_td3_speed[reduce-overhead-backward] 1.7580ms 1.5243ms 656.0329 Ops/s
test_cql_speed[False-None] 17.8797ms 17.2853ms 57.8525 Ops/s
test_cql_speed[False-backward] 23.9793ms 22.7134ms 44.0269 Ops/s
test_cql_speed[True-None] 3.7208ms 3.3250ms 300.7544 Ops/s
test_cql_speed[True-backward] 5.9538ms 5.7612ms 173.5751 Ops/s
test_cql_speed[reduce-overhead-None] 21.6269ms 13.5338ms 73.8893 Ops/s
test_cql_speed[reduce-overhead-backward] 2.1285ms 1.9419ms 514.9627 Ops/s
test_a2c_speed[False-None] 3.4030ms 3.2273ms 309.8599 Ops/s
test_a2c_speed[False-backward] 6.9174ms 6.3473ms 157.5471 Ops/s
test_a2c_speed[True-None] 1.5386ms 1.3616ms 734.4554 Ops/s
test_a2c_speed[True-backward] 3.2375ms 3.0605ms 326.7444 Ops/s
test_a2c_speed[reduce-overhead-None] 16.4831ms 9.1846ms 108.8778 Ops/s
test_a2c_speed[reduce-overhead-backward] 1.5680ms 1.4766ms 677.2406 Ops/s
test_ppo_speed[False-None] 4.1212ms 3.7669ms 265.4706 Ops/s
test_ppo_speed[False-backward] 7.4133ms 6.8976ms 144.9779 Ops/s
test_ppo_speed[True-None] 1.5848ms 1.4227ms 702.8991 Ops/s
test_ppo_speed[True-backward] 3.2312ms 3.0885ms 323.7854 Ops/s
test_ppo_speed[reduce-overhead-None] 1.1090ms 0.9680ms 1.0331 KOps/s
test_ppo_speed[reduce-overhead-backward] 1.5264ms 1.4246ms 701.9499 Ops/s
test_reinforce_speed[False-None] 2.4874ms 2.3228ms 430.5065 Ops/s
test_reinforce_speed[False-backward] 3.5393ms 3.3228ms 300.9483 Ops/s
test_reinforce_speed[True-None] 1.4856ms 1.3074ms 764.8557 Ops/s
test_reinforce_speed[True-backward] 3.1435ms 2.9743ms 336.2115 Ops/s
test_reinforce_speed[reduce-overhead-None] 18.7659ms 10.4089ms 96.0716 Ops/s
test_reinforce_speed[reduce-overhead-backward] 1.5800ms 1.4864ms 672.7725 Ops/s
test_iql_speed[False-None] 10.0792ms 9.4356ms 105.9819 Ops/s
test_iql_speed[False-backward] 13.4486ms 12.9627ms 77.1444 Ops/s
test_iql_speed[True-None] 2.4422ms 2.2513ms 444.1976 Ops/s
test_iql_speed[True-backward] 4.9275ms 4.7879ms 208.8601 Ops/s
test_iql_speed[reduce-overhead-None] 19.2343ms 11.3982ms 87.7335 Ops/s
test_iql_speed[reduce-overhead-backward] 2.0155ms 1.9026ms 525.5957 Ops/s
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.1084ms 6.6115ms 151.2515 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5319ms 0.2687ms 3.7210 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4755ms 0.2480ms 4.0326 KOps/s
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7064ms 6.3124ms 158.4192 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9261ms 0.2906ms 3.4408 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6442ms 0.2850ms 3.5092 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4809ms 1.2761ms 783.6164 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5692ms 1.1914ms 839.3700 Ops/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7872ms 6.5070ms 153.6797 Ops/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8727ms 0.4584ms 2.1817 KOps/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7075ms 0.4569ms 2.1889 KOps/s
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5612ms 6.3729ms 156.9152 Ops/s
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0413ms 0.2947ms 3.3928 KOps/s
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5233ms 0.2859ms 3.4983 KOps/s
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 9.3376ms 6.3655ms 157.0976 Ops/s
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6843ms 0.2675ms 3.7389 KOps/s
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5274ms 0.3110ms 3.2157 KOps/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7372ms 6.5129ms 153.5421 Ops/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1240ms 0.4354ms 2.2966 KOps/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6343ms 0.4294ms 2.3290 KOps/s
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.2551ms 5.6180ms 177.9984 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.1733ms 2.0984ms 476.5543 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.5117ms 1.1633ms 859.6247 Ops/s
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 10.1283ms 5.7683ms 173.3624 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 6.0895ms 2.0260ms 493.5806 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.9877ms 1.3018ms 768.1447 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4886s 15.5823ms 64.1753 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.4548ms 2.2541ms 443.6444 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.1057ms 1.3962ms 716.2332 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.6762ms 13.3469ms 74.9240 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.1986ms 17.4707ms 57.2388 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 19.1041ms 18.0369ms 55.4418 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.9665ms 17.9051ms 55.8500 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 19.3422ms 18.3867ms 54.3871 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.2231ms 18.8357ms 53.0908 Ops/s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot